1. Field
The embodiments discussed herein are directed to a semiconductor device and a method of controlling the same.
2. Description of the Related Art
Moving picture encoding schemes such as Moving Picture Experts Group (MPEG) have been widely used in various fields of digital broadcasting, Internet communications, mobile phone communications, and so on. In such moving picture encoding schemes, a decoder decodes bit stream data transmitted from an encoder and outputs the data. For example, when bit stream data is transmitted through a transmission line of radio communications, a factor such as a degraded state of communications causes an error in the bit stream data and the data including the error may be inputted to the decoder.
When such an error is detected by a decoder, the most reliable method for correcting the error is to request an encoder to resend encoded data corresponding to the error. However, making a request to resend data is not so practical in reality from various viewpoints such as keeping communication speed.
Consequently, when a decoder detects an error, the decoder has to perform a process to minimize the influence of the error. In an H.264 moving picture compression encoding scheme defined by the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) in 2003, a redundant slice is used to solve such a transmission error. Thus when an error occurs, a decoded image with a macroblock that includes the error can be recovered using the contents of a redundant slice.
When such a method cannot be used, a decoder may perform error concealment to minimize the influence of an error. Various techniques have been proposed for error concealment. Referring to FIGS. 16 to 25, a prior art technique of such error concealment will be described below.
FIGS. 16 and 17 illustrate explanatory drawings of defective blocks, which cannot be normally decoded, and image degradation blocks, which have been decoded but may have abnormal decoding results when an error is detected in a bit stream of a moving picture variable-length encoding scheme such as H.264. In FIG. 16, when the error is detected in block (macroblock) n+3 in the bit stream, the subsequent blocks before a recovery position are regarded as defective blocks that cannot be normally decoded, in consideration of the characteristics of variable length encoding. For example, when block n includes a defective bit, that is, a defective bit having caused the error detected in block n+3, there is a high probability that the picture quality of previously decoded blocks, for example, n to n+2 blocks may also be degraded.
In FIG. 17, blocks between (macro) block n+3, where the error has been detected, and the recovery position are regarded as defective blocks which cannot be normally decoded. For example, blocks n to n+2 are regarded as image degradation blocks which have been decoded but are assumed to have abnormal decoding results.
FIG. 18 illustrates an explanatory drawing of blocks where picture quality is degraded by the influence of the image degradation blocks through deblocking filtering, that is, filtering for reducing block deformations occurring on the boundaries of the blocks during encoding of an image. The following will examine the blocks which are adjacent to the image degradation blocks and have undergone deblocking filtering when the error is detected in error detection block n+3. In this case, it is assumed that picture quality is degraded also in blocks located above blocks n to n+3. In the error concealment of a decoder, it is necessary to identify blocks where picture quality is presumed to be degraded and perform error concealment on the identified blocks as on defective blocks.
FIG. 19 illustrates a structural block diagram of a moving picture decoder according to a prior art example. In FIG. 19, the decoder is made up of a control unit 100 for controlling the overall configuration, a decoding unit 101 for decoding input stream data, a frame memory 102 for storing decoded image data, a memory controller 103 for controlling exchange of image data between the decoding unit 101 and the frame memory 102, and an output unit 104 which reads, through the memory controller 103, image data having been decoded and stored in the frame memory 102, and outputs the image data as an output image.
The decoding unit 101 includes a variable length decoding section 111 for performing variable length decoding on input stream data, an inverse quantization/transformation section 112 for performing inverse quantization and inverse transformation on the output of the variable length decoding section 111, an intraframe prediction section 113 for performing intraframe prediction when an intraframe prediction scheme is used as an encoding scheme, an interframe prediction section 114 for performing interframe prediction when an interframe prediction scheme is used, a deblocking filtering section 115 for performing the foregoing deblocking filtering on decoded image data, and an adder 116 which adds the output of the inverse quantization/transformation section 112 and one of the outputs of the intraframe prediction section 113 or the interframe prediction section 117, which are switched by a switch 117, and supplies the added output to the deblocking filtering section 115.
When an error is detected in input stream data by the intraframe prediction section 113 and the interframe prediction section 114, error concealment is performed. Various schemes are available for error concealment. As one of the schemes, the following error concealment technique is available: error concealment reference images corresponding to an error detection block, defective blocks, or image degradation blocks are selected from image data having been decoded and stored in the frame memory 102, and the selected images are used instead of the image data of the image degradation blocks or the defective blocks.
In FIG. 19, when an error is detected while the intraframe prediction scheme is used as an encoding scheme, the switch 117 is switched to the interframe prediction section 114. Next, the error concealment reference images used instead of the data of defective blocks or image degradation blocks are read from the frame memory 102 through the memory controller 103. The image data of the images is added to the output of the inverse quantization/transformation section 112 by the adder 116, and is supplied to the deblocking filtering section 115. The output of the deblocking filtering section 115 is stored in the frame memory 102 through the memory controller 103 as decoding result image data corresponding to the image degradation blocks or the defective blocks. Also, when the interframe prediction scheme is used, the data of the error concealment reference images is similarly supplied to the adder 116 from the interframe prediction section 114.
FIGS. 20 to 22 illustrate control flowcharts performed by the control unit 100 of FIG. 19. FIG. 20 illustrates the flowchart of control performed on the decoding unit 101. In FIG. 20, processing on the decoding unit 101 is suspended in step S101 until free space becomes available in the frame memory 102. When free space becomes available, in step S102, the control unit 100 causes the decoding unit 101 to perform decoding on one screen, that is, decoding on one frame. In step S103, the control unit 100 manages reference images based on standards and specifications. For example, the control unit 100 performs management and so on when decoded images are used as reference images of other images in the interframe prediction scheme. After that, the processing from step S101 is repeated.
FIG. 21 illustrates the flowchart of control performed on the output unit 104. In FIG. 21, first in step S105, the control unit 100 causes the output unit 104 to wait for the output start timing, that is, the timing for reading output image data from the frame memory 102. At this timing, in step S106, an output image to be outputted from the frame memory 102 is determined based on the standards and specifications. In step S107, one-screen output image data read from the frame memory 102 through the memory controller 103 is outputted to the output unit 104. After that, the processing from step S105 is repeated.
FIG. 22 illustrates the flowchart of area release control in the frame memory 102. In step S108 of FIG. 22, the control unit 100 waits for an image which may be released based on the standards and specifications, that is, an image which may be deleted from the frame memory 102. In step S109, an area occupied by the image on the frame memory 102 is released. After that, the processing from step S108 is repeated. Whether an area may be released or not is determined by whether or not other images are unlikely to refer to the image data of the area when the image data has been outputted through the output unit 104 or when, for example, the interframe prediction scheme is used.
To explain a problem of the moving picture decoder of FIG. 19, the following will describe an example of a group-of-picture (GOP) structure indicating the decoding order and output order of image data in an input stream in accordance with FIG. 23. Referring to FIGS. 24 and 25, the following will describe a prior art example of the timing chart of decoding and output of image data according to the example of FIG. 23. FIG. 23 illustrates a GOP structure generally used in moving picture encoding schemes such as MPEG2 or H.264, illustrates the decoding order and the output order of image data in an input stream, and illustrates a reference relationship between images.
In other words, the upper part of FIG. 23 illustrates images in the input stream, that is, the decoding order corresponding to the order of pictures. The reference relationship is established as follows: an I0 picture first inputted to the GOP structure is referenced by pictures B1, B2, B4 and B5, which are subsequently inputted as B pictures, and a P3 picture inputted as a P picture, where the P3 picture is referenced by pictures B4, B5, B7 and B8 and a P6 picture, and the P6 picture is referenced by the B7 picture and the B8 picture. In FIG. 23, the pointed ends of arrows represent the referring pictures and the round ends of the arrows represent the referenced pictures.
In the picture output order of the lower part of FIG. 23, the decoding results of the B1 and B2 pictures are first outputted, and then the decoding result of the first inputted I0 picture is outputted. After that, the decoding results of the B4 and B5 pictures are outputted, and then the decoding result of the P3 picture is outputted. Further, the decoding results of the B7 and B8 pictures are outputted, and then the decoding result of the P6 picture is outputted. In this way, the output order of the decoding results is different from the order of pictures in the input stream.
Generally, when an error is detected in an input bit stream as described above and error concealment is performed also on image degradation blocks preceding an error detection block, the influence of the error on an output image can be made less noticeable by performing error concealment as reprocessing on the decoded image degradation blocks. However, reprocessing increases an amount of operations. For example, it is thought that when quite a large number of errors occur in an image (picture), the decoding time may exceed a decoding time allocated to the decoding of image data of the image because the amount of operations is increased by reprocessing.
The timing chart of FIG. 24 illustrates an example of a timing chart of decoding and outputting when errors occur in the P3 and B4 pictures of the pictures in the input stream of FIG. 23 and the images include defective blocks and image degradation blocks caused by the errors. In this example, in the decoding of the P3 picture, which is an image including the error, the I0 picture is referenced for error concealment. In the decoding of the B4 picture, the P3 picture as a result of the error concealment is referenced and error concealment is performed on the B4 picture. Further, the decoding of these two pictures requires a longer time than the decoding of other pictures. However, since the I0 picture is outputted between the B2 picture and the B4 picture, the output is correctly performed as illustrated in FIG. 23 and the decoding time increased by the error concealment does not affect the output of image data. In the lower part of FIG. 24, banks 0 to 3 represent areas in a frame memory for storing the data of a single image, that is, a single picture. In this case, when the output of data stored in each bank is completed and other pictures are unlikely to refer to the data, the data is deleted from each bank as illustrated in FIG. 22.
FIG. 25 illustrates an example of failed processing in which the decoding time of the P3 and B4 pictures exceeds the decoding time of FIG. 24 and image data output is different from that of FIG. 23. In FIG. 23, the P3 picture is decoded until the end of the output of the B2 picture and the decoding time of the B4 picture from that time is increased, so that the output of the B4 picture is late and the I0 picture is outputted twice. Consequently, the processing time for the image data output of the overall GOP configuration of FIG. 23 is increased by one picture.
As described above, in the prior art, when error concealment is performed on image degradation blocks having been decoded before an error detection block, a picture including quite a large number of errors may have a decoding time exceeding a predetermined processing time.
Japanese Patent Laid-Open No. 2001-119693 relating to error concealment in a moving picture decoder discloses a technique in which in the event of an error in decoded image data, irregularities on a displayed image are made less noticeable by simple error concealment of repeatedly displaying a currently displayed image.
Further, Japanese Patent Laid-Open No. 2004-320252 discloses a technique of reducing an amount of RGB conversion when a displayed image is generated from a decoded image in the event of an error during the decoding of the image.
However, such a prior art technique cannot solve a problem of a decoding time of an image, that is, a picture. The decoding time may exceed a predetermined processing time when error concealment is performed as reprocessing on image degradation blocks and so on which have been decoded before an error detection block.